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ABSTflACT 

issues pertaining to rescaich on college reading and 
study skills programs are analyzed, based on a literature review* It 
is suggested that before educators can provide quality instruction, 
they need to differentiate practices that are conducive to learning* 
Evaluative criteria are needed to make such decisions, as well as to 
design and evaluate experimental investigations* Particularly 
important are the following: (1) sampling considerations, including a 
represent at ive study populat ion , randomizat ion , analysis of 
covariance , and internal validity ; (2) independent variables ; ( 3 ) 
training in the teaching technique under analysis; (4) instriiments 
that are adequately described and appropriate to the task; and (5) 
data analysis techn iques that are appropr iate for the experimental 
design* To bridge the gap betii^een practice and theory, cooperative 
research ventures between instructors and researchers must be 
undertaken * It is suggested that college reading staff need to: 
understand the contributing role of research, develop the skills 
necessary for evaluating and conducting studies, and continue to seek 
the institutional support (release time. Seed money, assistants) 
necessary for conducting applied research* (SW) 
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''Evaluative Criteria for College Reading-Study Research" 

As a general rule, much of the instruction which occurs in 
applied learning settings tends to be based on either long- 
standing conventional wisdom or questionable resear ch 
methodologies. By no means is the fie Id of reading an exception 
to this rule, Harris (1976) states, for instance, that there is 
no agency which protects either the reading teacher or pupil from 
educational practices that are based on either poor research or 
unsubs ta ntiated tradition* Nor is either group protected from 
so- ca 1 1 ed in no va ti ve practices vh ich have gained undue wide 
acceptance through extensive media coverage, followed by a 
band wagon effect* Certainly soiree reading re sear ch has had a 
positive impact on instructional practices in the classroom 
(Russell, 1961), but there also exists a body of published 
research that has had greater influence than it actually warrants 
(s inger, 1970)* Does the research relating to co liege reading 
and study- skills programs fall in the former or latter category? 
For college reading and study-skills educators, the ability to 
differentiate those practices wh ich are truly conducive to 
learning from those which are not is absolutely critical to 
providing quality instruction* 
What Does the Literature Suggest? 

Two thorough reviews of the col lege reading and study- skil 1 s 
literature (Sanders, Lowry, & Theimer, 1982; Stahl, 1983) indi- 
cate that despite major methodological and reporting flaws, 
research has indeed been translated into practice* In Stahl's 



(1983) critical ana 1 ys is of the research op te x ^-book - study 
methods advocated in college reading programs^ he concluded that 
virtually all of nhe identified investigations suffered from 
Serious problems with experimental design/ graining procedures, 
measurement devices, and statistical procec7ures* The extent of 
these difficulties made it impossible to reach any firm conclu- 
sions about the relative effectiveness of textbook -study syst*^ms* 
On a larger scale, Sanders/ Lowry/ and Thf^imer (1982) presented a 
comprehensive analysis of the literature concerning the effec^ 
tiveness of college reading and study-skil Is programs* in their 
meta-ana lysis of the relevant research from 1960 to 1977 (see 
Sanders, 1979/ for the total study), they identified a total of 
676 studies* Of these, on ly 66 repcrted data in quantifiable 
terms and only 28 of the studies were sufficiently controlled to 
warrant inclusion in the meta-analysis* Appropriately, these 
researchers conclude that serious de f iciencies in reporting what 
actually transpired in college reading programs make it very 
difficult for practitioners/ program developers/ and investi^ 
gators to know what variables contribute to reading growth. 

As a result, it is imperative that the search continue for 
the variables that increase skills if systematic and general 
improvement in college reading-study instruction is to occur* 
For col lege reading- study personnel , this search should progress 
in two basic directions* First, they must become better 
a cqua inted winh the rele van t literature wh ich m ay i mpac t upon 
their practice* This endeavor translates into regularly 
surveying journal articles/ dissertation abstracts/ and new 
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texts, as we LI as attending proftissional meetings, Addi t ionci II 
i ndi vidua 1 s whc oversee co L Lege reading and study~?^kiLLs programs 
must provide encouragement and material support tor these 
activities if variables that contribute to program success are 
ever to be known* Second, co L lege readi ng per sonne L must beconin 
researchers them se L ves* Rg Liance on past practices by v Irtue of 
tradi cion , popuLdr i ty, or presumed data-driven support is tenuous 
at best. Furthermore, each instructionaL situation is somewhat 
unique unto itself. Consequently, di s cover i ng key variables 
related to the success of the immediate program beco rn es the 
responsibility of the indi vj dual adm in i storing that program* 
Essential Criteria 

Before reading personnel can accurately and adequately 
evaluate rasoarch studies for the purposes of determining 
variables of success or as a preliminary step in their own 
investigations, they must possess a set of accepted Evaluative 
criteria or standards* Wallen (1974) points out that in 
reviewing and evaluating a research study an investigator should 
utilize the same standards that are important for designing one. 
Such a view is central to the point of this article* Several 
reputable sources on educational research (Best, 1977; issac & 
M ichael , 1976; Tuck man, 1972) and also on readi ng research 
(Clymer, 1969; Harris, 1976; Kingston St Weaver, 1967) advocate 
using similar criteria for designing and, hence, following 
Wallen ' s logic , for evaluating experimental investigations * 

Wh i 1 e it is r ecom me nded that college reading and s t udy- 
ski lis personnel consult the above sources for an exhaustive list 



of evaluative criteria, there are several criteria that are 
pa r t icularly im por tan t in analyzing research r epor t s about 
college reading and study-skills pro grams* Special emphasis 
needs to be placed on these criteria since they appea r to be 
overlooked regularly by those conducting research with college 
reading populations , 
Sampl ing 

Several points need to be considered to ensure tha t the 
sample is representative of the population* First, it is highly 
desirable for members of the sample to be randomly assigned to 
all treatment or control groups rather than assigned through 
matching students on cne or more variables* When randomization 
is impossible, the appropriate statistical technique, the 
analysis of co variance (see Cox, 1958; Kirk, 1968) should be 
utilized to approach the effect of randomization* Related to 
this point is the control ling for particular factors which 
jeopardize internal validity (i*e*, whether or not the treatments 
actually made a difference)* The main threat to internal 
validity with groups that are not randomized arises from the 
possibility that some critical difference, not reflected in pre- 
testing, exists to ccn taminate post test data* For example, if 
one treatment group is from an evening class, the students in 
that group may be older and perhaps more motivated than a morning 
class of 18-year-olds j ust out of high school* 

Two sources of invalidity are of pa r t i cu lar con ce rn in 
r evi ew ing studies on the success of college reading^study 
programs! statistical regression, the shi f t toward the mean on 



repeated or corrt; la ted measures, and experimental mortality, the 
differential loss of participants from the comparison* 
Statistical regression to the inean is of concern because a great 
many of the students who enroll in col lege study- ski 1 Is classes 
are oercei ved to be renedia 1 students because of low high school 
G*P.A*'s or SAT scores, or are advisee^ xnto such courses because 
of very low scores on entrance examinations* Ther e f or e , it is 
highly probable that the mean score of those students who score 
exceptionally low will move toward the average of the total group 
whether or not a treatment is applied* Second ly, this population 
is more apt to drop out of college or have erratic attendance 
patterns * 

A second major point pertains to the independent variables 
which are under study* The variables must be sufficiently 
different and sufficiently powerful to lead to different results* 
For example, one cannot expect, two methods of studying that are 
only slightly different (s tudy-sk il Is methods tend to have ruore 
commonalities than differences) to lead to significant 
differences* On the other hand, there is a danger in constantly 
comparing each new me thod for studying against a simple read- 
reread technique * 

Since generalizations from this type of study tend to be 
transferred to an applied setting, the procedures for training 
should be adequate for the task, and the goals and objectives of 
the ir structional methods should be clearly defined* One of the 
essential character istics of a good research study is that it 
should be replicable* By clearly specifying methods and 



objectives, the chances are improved for replication and 
increased general izabi 1 ity of findings* 

Since most stud y-ski lis techniques under investigation are 
new to the subjects, they must receive training of a duration 
long enough so as to insure understanding and internalization of 
the technique * This is particularly important when a technique 
is compared to a "tried and true" method (read^reread or under^ 
1 in ing ) wh i ch a student may have been using for many years 
previous to the investigation* On the other hand, without long 
term training, an initial gain in reading may actually be based 
on the novelty of using a new technique* As an example, students 
often double their reading rate after attending the introductory 
session of a dynamic speed reading course* This gain, however, 
can be linked to a novelty effect since the increase is not 
likely to be maintained over time without long term training that 
leads to mastery of proven rate development techniques* 
Instruments 

The instruments utilized in measuring the effect of the 
tratment should be adequately described, and must be appropriate 
for the task* Clymer (1969) and Harris (1976) both note that the 
sophistication of the analysis cannot make up for the use of 
either poor instruments or the wrong instruments* As an example, 
a standardized norm-referenced test cannot be expected to measure 
the effectiveness of a te xtbook-study system because of the 
instrument's survey level design and its timed nature; yet, 
several researchers have utilized just such an instrument in 
their work* 



Analysis of Data 

In analyzing the data from an investigation, the research 
should employ and report the appropriate statistical techniques 
for the nature of the experimental design* From the data the 
researcher should attempt to determine the sign ifi cant relation- 
ships and also to provide a logical, accurate analysis* It is 
not necessary for a study to lead to significantly different 
results to imply practical applications* Yet inferences drawn 
from the study should be di f f erentiated from scienti f ic findings 
There is danger in " fishing for trends" or extended rationalizing 
when signi f icance is not found between groups* This temptation 



is particularly great when the researcher is evaluating a method 
that has been personally developed or is part of an ongoing line 
of research * 

While it is relatively easy to differentiate between a 
superbl y- conducted study and a tot a 11 y i nad equate one , it is not 
nearly as easy to evaluate those which are only partially flawed* 
One method is to follow a basic set of criteria for good research 
with special attention being given to potential problem areas 
associated with a particular line of research, as in the case of 
college reading^ study programs* 

The Prospects of Future Research on Colleg^e Reading Programs 

The prospects of ongoing research with instructional methods 
and reading-'Study techniques used in college reading and study- 
skil Is courses and in learning skills centers is questionable if 
the past is any indication of the future* Over the years 
educators associated with college reading programs have stressed 
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both the development of instructional materials based on 
tradition or current fads and the direct interaction with pupils 
over formal research endeavors* such a sit-.uation exists/ in 
part/ because of the limited background of many college reading 
specialists in the foundations and methods of educational 
research* This factor is coupled with a widespread/ fundamental 
belief that all students, whether they are taking developmental 
courses or receiving learning center services, snoul d never be 
denied assistance that might support their success in obtaining a 
college education* Hence, the traditional design of empirical 
research which may withhold services is often viewed as 

in com pa t ibl e \v i th the ph i losophical tenets and the i nhe r en t 
mission of college reading and study-skills programs* Therefore, 
while in practice college reading personnel may welcome the 
introduction of new instructional methods* they tend to disfavor 
the use of control groups and rigorous adherence to experimental 
procedures * 

When research is performed, all too often tne investigators 
conducting studies with lower-division students at community 
colleges, liberal arts colleges, or comprehensive universities 
come from prestigi ou s universities* These r e s ear cher s do no t 
necessarily understand the educational and developmental 
characteristics of the population enrolled in college reading 
classes* While the search for new methods and strategies may 
utilize the latest theory, the resultant new ideas and 
"educational advances" can be as unwieldy and unacceptable to the 
intended clientele as the current systems * 
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Summary 

Why haven^t the researchers who wrote dissertations on 
topics such as the effectiveness of textbook- study systems under- 
taken further research with college reading instruction? The 
vast majority of doctoral candidates completing dissertations in 
the field of college reading either never publish again or turn 
to more immediately productive areas of inquiry* As pointed out 
by Maxwell (1964) and Gordon and Flippo (1983), the reason for 
this trend is not entirely clear* 

How then can future research endeavors with college reading- 
study strategies be conducted in such a way as to bridge the gap 
between practice and theory, instructor and researcher? In the 
short run, cooperative research ventures between qualified 
investigators and informed practitioners must be undertaken* In 
the future, however, developmental educators must accept that the 
very existence of college reading programs is dependent upon the 
presentation of overh proof that programs and instructional 
components help students meet their educational goals* This task 
will require co liege reading pe rson ne 1 ( 1 ) to under stand the 
contributing role oJ: research, (2) to develop the skills 
necessary for evaluating and conducting studies, (3) to continue 
to conduct research upon completing the terminal degree, and (4) 
to seek actively the institutional support (release time, seed 
money, assistants, secretarial services, etc*) necessary for 
conducting applied research* Then, investigations may prove 
fruitful in the generation of data that not only advance theory 




but also promote the utilization of proven instructional methods 
for the college reading program. 
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